REPLACEMENT SPECIFICATION 
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VEHICLE FRONT-VIEW MONITORING SYSTEM 

BACKGROUND OF THE INVENTION 
The present invention relates to a vehicle front-view monitoring system having a function of 
5 taking fail-safe measures against images irregularly monitored due to sunlight interfering. 

The vehicle front-view monitoring system having CCD cameras has recently attracted much 
attention. The system recognizes driving conditions such as a distance to a vehicle travelling ahead based 
on images taken by the cameras and warns a driver of danger or controls the vehicle such as by downshifts. 
Recognition of the driving conditions using a stereo-distance measuring technique obtains a 
10 positional difference (parallax) on an object in a pair of the images and calculates the distance to the 
vehicle travelling ahead by trigonometrical survey. 

Such a vehicle front-view monitoring system requires fail-safe measures for insuring safe driving, 
A failure to be detected by the system is a fault monitoring condition such as when the vehicle in front is 
beyond recognition while monitoring cameras are facing weak sunlight. 
1 5 Image monitoring failures are discussed with reference to FIGS. 1 to 3. 

When monitoring vehicles that are traveling ahead (FIG. 1) while facing weak sunlight, disc-like 
difiusion patterns could appear on the monitored image, as illustrated on the left-upper side in FIG. 2, due 
to saturation of luminance. Such a disc-like difiusion pattern spearing on the monitored image while the 
vehicle is facing the sunlight is called a sunlight-difiiision disc hereinafter. These sunlight-difiiision discs 
20 cause the saturation of pixel luminance on the disc areas and the surrounding areas. A correct image data 
therefore can not be obtained, or vehicles in front are masked by the sunlight-difiiision discs, which results 
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FIG. 3 illustrates distance-data calculation based on stereo images (a reference image and an 
image to be compared such as shown in FIG. 1). 

In FIG. 3, black sections have a big luminance change between the adjacent pixels in the 
horizontal direction. These sections carry distance data (depth). The intensities of most pixels in the 
5 sunlight-difiiision disc area have been saturated, so that no distance data is calculated in this area (a lost 
state). Moreover, the disc edges have a big luminance change in the horizontal direction, so that the 
incorrect distance data could be calculated. 

Insuring safe driving in such a fault condition due to a generation of the sunlight-difiiasion discs 
discussed above requires the fail-safe measures to temporarily halt the monitoring fimction. 
10 SUMMARY OF THE INVENTION 

A purpose of the present invention is to provide a vehicle front-view monitoring system for 
monitoring optical irregularities occurring on monitored images and swiftly taking fail-safe measures. 

Another purpose of the present invention is to provide a vehicle fiont-view monitoring system 
taking appropriate fail-safe measures while avoiding repetition of unnecessary fail-safe measures. 

The present invention provides the vehicle front-view monitoring system for taking the fail-safe 
measures when a fail-safe measure-interruption requirement using a first parameter is met on a monitored 
image for a predetermined first period and resuming a fimction intermpted by the fail-safe measures when 
a fail-safe measure-release requirement using a second parameter difiFerent Scorn the first parameter is met 
within a predetermined second period after the fail-safe measure-intermption requirement has been met. 

Moreover, the present invention provides a vehicle front-view monitoring system for taking a 
fail-safe measure when a fail-safe measure-intenuption requirement is met on a monitored image for a 
predetermined first period and resuming a fimction interrupted by the fail-safe measures when a fail-safe 
measure-release requirement is met within a predetemiined second period after the fail-safe 



measure-interruption requirement has been met, the first period being variable in accordance with an 
accuracy of a lane marking on a road in the monitored image is recognized. 

BRIEF DESCRPTION OF DRAWINGS 
FIG. 1 illustrates a normal image monitored on a monitor screen; 

FIG. 2 illustrates an irregular image monitored on the monitor screen in which sunlight-diffiision 
discs are shown; 

FIG. 3 illustrates the irregular image monitored on the monitor screen, including inaccurate 
distance data due to generation of sunlight-difiiision discs; 

FIG. 4 is a block diagram of a preferred embodiment of the vehicle front-view monitoring 
system according to the present invention; 

FIG. 5 shows a flow chart of a weak-sunlight-facing driving determination routine; 

FIG. 6 shows a flow chart of a weak-sunlight-facing driving detennination routine; 

FIG. 7 illustrates the upper and lower areas set on the monitor screen; 

FIG. 8 illustrates an area in which a vehicle travelling ahead is monitored; 

FIG. 9 illustrates a monitoring area; 

FIG. 10 illustrates measured luminance-distribution characteristic parameters; 
FIG. 1 1 illustrates normalized luminance-distribution 
characteristic parameters; 

FIGS. 12A and 12B illustrate luminance addition characteristics in noraial driving; 

FIGS. 13A and 13B illustrate luminance addition characteristics in weak-sunlight-facing driving; 

FIGS. 14A to 14C illustrate luminance center; 

FIG. 15 shows a flow chart of a fail determination routine; 

FIG. 16 shows a flow chart of a count-up amount setting routine; 
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DETAE.ED DESCRIPTION OF PREFERRED EMBODIMENTS 
Preferred embodiments according to the present invention will be disclosed with reference to the 
attached drawings. 

5 FIG. 4 shows a block diagram of a preferred embodiment of a vehicle font-view monitoring 

system according to the present invention. 

A pair of cameras 1 and 2 having image sensors such as CCDs are attached to a vehicle such as 
an automobile in the vicinity of tires with a predetermined distance (corresponding to camera-baseline 
length) in the direction of vehicle width, for taking images ahead. 
1 0 The main camera 1 (right camera) takes a reference image required for stereo processing whereas 

the sub-camera 2 (left camera) takes an image to be compared. 

Analog images taken by the cameras 1 and 2 while the main camera 1 and sub-camera 2 are 
synchronized with each other are converted into a specific luminance gradation (for example, gray scale at 
256 gradations). 

1 5 The digitized images are processed by an image correcting section 5 for correction of luminance, 

image geometrical conversion, etc. An inevitable misalignment on the cameras 1 and 2 will aflfect left 
and right images. Such effect can be cancelled by geometrical conversion such as an image rotation and a 
parallel shift with Affine conversion, etc. Luminance data on the stereo images (a reference image and an 
image to be compared) for one fi:^e corrected by the image correcting section 5 are stored in an image 

20 memory 8. 

The luminance data are fiuther supplied to a stereo-image processor 6 for calculating distance 
data (parallax for each object). Each distance data corresponds to a positional deviation on the same 
object in the stereo images. An actual distance to the object is defined on the basis of parallax by a 
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trigonometrical survey. The distance data for one frame thus obtained are stored in a distance-data 
memory 7. 

The luminance data and the distance data are supplied to a microcomputer 9. 

The microcomputer 9 includes a road-recognizing section 10, a three-dimensional (3-D) 
5 object-recognizing section 1 1 , a fail-determining section 1 2 and a processor 1 3 . 

In response to the luminance and distance data, the road-recognizing section 10 recognizes the 
condition of a road ahead, and the 3-D object-recognizing section 1 1 recognizes three-dimensional objects 
(vehicles travelling ahead). 

Based on the data from the sections 10 and 1 1, the processor 13 warns the driver of a danger via 
10 a warning device 1 9 such as a display or a speaker when it determines that the warning is necessary. 

Moreover, the processor 13 controls several controllers 14 to 18 if necessary. For instance, it 
sends a command signal to an automatic transmission (AT) controller 14 for dovmshifts or to an engine 
controller 18 for an engine power-dovm. The processor 13 may further sends the command signals to an 
antilock-braking system (ABS) controller 15, a traction control system (TCS) controller 16 and a vehicle 
1 5 controller 17 for controlling an engine speed and a torque distribution to tires, and etc. 

The failure-determining section 12 determines whether the cameras 1 and 2 are taking the images 
into the sun and a failure is occurring, based on the luminance and the distance data stored in the image 
memory 8. The vehicle and the warning controls are intemipted by the fail-safe measures for prevention 
of malfrinction which could otherwise occur due to lowering in monitoring accuracy during a period of 
20 determination in that a failure is occurring, or of "1" set for a failure flag NG which v^ll be described later. 

FIGS. 5 and 6 show flowcharts for indicating a routine of determining whether the cameras 1 and 
2 are taking images into the sun while the vehicle is facing the sunlight, which is called a sunlight-facing 
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driving hereinafter. The routine is repeated by the failure^ietermining section 12 by a predetemiined 
interval (a cycle of 100 ms for example). 

The requirements for taking the fail-safe measures (called fail-safe measure-intemiption 
requirements hereinafter) are discussed before explaining the routine in detail. 
5 Requirements for steps SI to S6 are necessary for taking the fail-safe measures in the present 

invention. It is determined that the cameras 1 and 2 are taking the images into the sun only when all the 
fail-safe measure-intermption requirements are met in steps SI to S6 and the routine proceeds to step SI 2 
whereas to step S7 if not met in any one of steps SI to S6. 

As disclosed later, the failure determining section 12 determines whether the fail-safe 
1 0 measure-intermption requirement or a fail-safe measure-release requirement is met and performs counter 
increment or decrement in accordance with a result of determination. The fail-safe measure-release 
requirements are the requirements for resuming fiinctions that have been intenupted by taking the fail-safe 
measures. 

The intermption requirements are required for determining whether the cameras 1 and 2 are 
1 5 taking images into the sun, so that the fail-safe measures should be taken. A counter increment is 
performed when an intermption requirement is met. The fail-safe measures will never be taken before the 
counter reaches a predetermined value, even if the intermption requirements are met. 

The release requirements are required for determining whether to halt the counter increment, hi 
the present invention, the counter starts a count-up when an intermption requirement is met and continues 
20 until a release requirement is met even though monitoring situation is changing and no intermption 
requirement is met anymore. 
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The present invention deploys the following six intemiption requirements for taking fail-safe 
measures, however, any combination of at least two of the requirements or addition of other requirements 
are available. 

First Requirement: No vehicle is recognized ahead, or no distance to a vehicle in Scont is 
5 calculated (step SI). 

The fail-safe measures will be taken in the present invention when the cameras 1 and 2 are taking 
the .images into the sun (sunlight-facing driving condition) so that the vehicle that is actually traveling 
ahead is lost on a monitor screen of the monitoring system. In other words, the fail-safe measures will not 
be taken even though the sunlight-diffusion discs shown in FIG. 2 are spearing when the vehicle traveling 
1 0 ahead is recognized. 

The present invention th^efore deploys a decision in that no vehicles are recognized ahead as a 
fail-safe measure-intermption requirement. This decision step is executed even though no vehicles 
actually exist ahead. 

When the requirement in step SI (FIG. 5) is met, the routine proceeds to step S2 to determine 
1 5 whether the cameras 1 and 2 are taking the images into the sun (sunlight-facing driving condition). 

Calculation of the distance to the vehicle in front in step SI is explained before disclosing the step 

S2. 

The distance to the vehicle in front is calculated in combination of a stereo-distance measuring 
technique and an image-recognition technique. 
20 Calculated first is a lane model that indicates a three-dimensional shape of a road ahead. Lane 

markings painted on the left and right sides of a lane and displayed on the monitor screen have the 
luminance higher than that of the road surface. The markings are extracted as areas having luminance 
edges on the borders between that and the road surface. 
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The parallax on the lane thus extracted is calculated for each (4 x 4)-pixel block, for example. A 
two-dimensional position and the parallax in the monitored image for one pixel block in which the lane is 
displayed provide a three-dimensional position and a distance of the lane portion by using known 
coordinate conversion formulas. 
5 The calculated three-dimensional lane distribution is ^proximated into a straight line for a 

predetermined segment. The straight lines are connected into a polygonal line for calculating the lane 
model. 

The vehicle travelling ahead is recognized as a three-dimensional object existing on the area 
(road surface) sun-ounded by left and right lane models. The distance to the vehicle in front is calculated 
1 0 based on the parallax of the object. 

These techniques are disclosed in detail in J^anese Patent Application No. 11-269578 (road 
recognition) and Jq^anese Unexamined Patent Publication No. 5-265547 (recognition of vehicle traveling 
ahead). 

Second Requirement: A shutter speed "a" is equal to or lower than a predetemiined reference 
1 5 speed (step S2). 

The shutter speed tends to be higher in a sunlight-facing driving condition where the vehicle in 
front is not visible on the monitor screen than that in a usual driving condition. In view of this tendency, 

an appropriate reference shutter speed such as 2000 jj sec is set 

The present invention deploys a decision in that the shutter speed "a" is lower than this reference 
20 shutter speed as a fail-safe measure-intemiption requirement. 
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The shutter speed, however, tends to be extremely higher against a strong sunlight in 

mid-summer, for example 200 |J sec. or lower. In view of this tendency, the routine proceeds to step S3 

(FIG. 5) when the requirements in steps S 1 and S2 are both met. 

Third Requirement: The upper luminance saturation factor LRl is larger than the lower 
5 luminance saturation factor LR2 (step S3). 

Sunlight-diflusion discs images shown in FIG. 2 v^ll be generated on the upper side of the 
monitor screen when a vehicle in fix)nt is not visible during sunlight-facing driving. The tendency for this 
phenomenon is because the luminance saturates on the upper side. 

In view of this tendency, as illustrated in FIG. 7, the upper and the lower areas are provided in a 
1 0 reference image. 

The present invention deploys the decision in that the upper luminance saturation factor LRl in 
the upper area is larger than that of the lower luminance saturation factor LR2 in the lower area as a 
fail-safe measure-intenuption requirement. 

The luminance saturation factors LRl and LR2 correspond to a ratio of the number of pixels for 
1 5 which luminance has saturated (for example, the luminance 240 or more) to the total number of pixels in 
the upper and the lower areas, respectively. 

The requirement in step S3 could be met even though no sunlight-diflfiision discs (FIG. 2) are 
generated when there is no shielding on a fair day or while travelling on a bridge. In view of this, the 
routine proceeds to step S4 (FIG. 5) when the requirements in steps SI, S2 and S3 are all met. 
20 No reference values are set for the upper and lower luminance saturation factors LRl and LR2 in 

this embodiment. This is because factors tend to vary very largely according to the driving conditions. 
Particularly, the lower factor LR2 often becomes zero in a normal driving condition. 
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Fourth Requirement: The number of luminance edges EN for an area of the vehicle in front is 
smaller than a predetennined reference number (step S4). 

While the vehicle travelling ahead is not visible on the monitor screen during the sunlight-facing 
driving, the number of the luminance edges EN on the sunlight-diffusion disc area (FIG. 2) tends to 
5 become extremely small. 

The number of the luminance edges EN indicates the number of a pair of adjacent pixels in the 
horizontal direction on the monitor screen for which variation in luminance is more than a predetermined 
reference value. 

As illustrated in FIG. 8, a vehicle in front area is set in the upper-mid section of the monitor 
1 0 screen, in which the vehicle travelling ahead would be displayed. 

The present invention deploys the decision in that the number of the luminance edges EN in the 
vehicle in front area (FIG. 5) is smaller than an q)propriate reference number as a fail-safe 
measure-intermption requirement. 

This requirement could, however, be met in driving at night, in tunnel, mist, smoke or on a 
1 5 snowy road. In view of this, the routine proceeds to step S5 (FIG. 5) when the requirements in steps SI, 
S2, S3 and S4 are all met. 

The number of the calculation of the distance data in the vehicle in front area can be used instead 
of the number of the luminance edges EN. This is because the number of the luminance edges EN and 
the number of the calculation of the distance data have a strong correlation in a stereo-matching, or the 
20 larger the number of the luminance edges EN, the larger the number of the calculation of the distance data. 

Fifth Requirement: The luminance-distribution- characteristics parameter normalized by using a 
shutter speed "a" meets a requirement. 
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As shown in FIG. 9, several slit-like monitoring areas Ni are arranged on the upper side of the 
monitor screen in the horizontal direction. The reason for the arrangement of monitoring areas Ni on the 
upper side is that the luminance mostly varies on the upper side when the vehicle is facing a weak sunlight. 

The monitoring areas Ni are provided for evaluating the luminance-distribution characteristics 
5 for a monitored image in the horizontal direction. 

For example, a reference image is divided into blocks of 16 pixels each in the horizontal direction 
when the reference image is composed of 200 and 512 pixels in the vertical and horizontal directions, 
respectively. Therefore, 32 monitoring areas Ni of (200 x 16) pixels are set. 

Calculated next is addition of luminance Ai for each monitoring area Ni. The addition of 
1 0 luminance Ai for a monitoring area Ni can be calculated by adding the luminance of sample pixels that are 
unifonnly distributed over the monitoring area Ni. The added luminance of the sample pixels may fiirther 
be averaged. 

The addition of luminance Ai is the value, for example, obtained by adding the luminance of 4 
pixels sampled in the horizontal and vertical directions each. Pixels uniformly distributed over the 

1 5 monitoring area Ni can be used for calculating the luminance over the area Ni with less amount of 
calculation. The addition of luminance Ai can be calculated for all pixels involved in the area Ni if the 
amount of calculation is out of question. 

The Ai-distribution characteristics for the 32 monitoring areas corresponds to the image 
distribution characteristics in the horizontal (lateral) direction. 

20 The parameter (luminance-distribution characteristic value) that indicates luminance-distribution 

characteristics on the addition of luminance Ai is normalized by a shutter speed "a". Used here as the 
parameter is luminance-addition variance VAR that indicates a variance in the addition of luminance Ai. 
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Luminance-addition variance VAR' normalized by the shutter speed "a", called a normalized 
luminance-addition variance VAR', is calculated by the following equations: 
VAR*-VAR/a 

VAR^={(A1-Aave)^ + (A2-Aave)^+ ... (An-Aave)Vn= I (AiVn-(I Ai/n)2 ... (1) 
The normalized luminance-addition variance VAR' as the luminance-distribution-characteristics 
parameter requires a decision in that the VAR' is larger than an ^propriate reference value as of fail-safe 
measure-intermption requirement. 

Instead of the VAR', the maximum addition of luminance Amax that indicates the maximum 
(peak) of the addition of luminance Ai can be used as the luminance-distribution-characteristics parameter. 

The Amax requires a decision in that the maximum addition of luminance Amax ' normalized by 
the shutter speed "a", called normalized maximum addition of luminance Amax', is larger than an 
appropriate reference value as a fail-safe measure-intermption requirement. 

Another decision in that both the nomialized VAR' and the normalized Amax' are larger than the 
con-esponding appropriate reference values can be used as a fail-safe measure-intermption requirement. 

The reason for using the normalized VAR' and Amax ' for determination of whether the vehicle is 
facing the weak sunlight is discussed with reference to FIGS. 1 0 and 1 1 . 

FIG. 10 shows the un-normalized characteristics of the luminance-distribution-characteristics 
parameters (VAR and Amax) that were measured in an actual driving test. Images were taken into the 
weak sunlight &om about 300 sec. to about 340 sec. after the start of measurement. 

The measurement of Amax teaches that Amax was measured in a fixed range through the test 
and hence it is difficult to find out an extreme tendency on a weak-sunlight-faciag driving. 
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The measurement of VAR shows a peak on about 270 sec. after the start of measurement, which 
appeared while the vehicle was not facing the sun, and hence it is also difficult to find out an extreme 
tendency on the weak-sunlight-facing driving condition. 

FIG. 11 shows the normalized characteristics of the luminance-distribution characteristics 
5 parameters (VAR! and Amax'). 

Compared to the test results shown in FIG. 10, the results in FIG. 1 1 show an extreme tendency 
in which both VAR' and Amax ' are extremely increased while the vehicle is facing the weak sunlight. 

It is therefore apparent that these parameters VAR' and Amax' can be used effectively for 
detennining whether the vehicle is facing the weak sunlight. 
1 0 Accordingly, the present invention employs the parameters Amax ' and VAR* that are obtained by 

normalizing the parameters Amax and VAR with a shutter speed "a" for determination of the weak 
sunlight-facing driving in view of the fact that Amax and VAR are the functions of the shutter speed "a". 

The decision in that both Amax ' and VAR' are larger than the corresponding reference values as 
a fail-safe measure-intoruption requirement fiirther enhances accuracy of determination on fail-safe 
1 5 measurements. 

The shutter speed, however, tends to be higher on a fair day, and hence the routine proceeds to 
step S6 (FIG. 5) when the requirements in steps SI, S2, S3, S4 and S5 are all met. 

Six requirement: Division of the luminance moment by the total addition of the luminance is 
equal to or lower than a reference value (step S6). 
20 Step S6 evaluates the distribution of addition of luminance Ai for each monitoring area Ni fi-om 

the point of view which is diflFerent fi"om step S5. 

FIGS. 12A and 12B show the distribution characteristics of the addition of luminance Ai in 
regular conditions. 
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In detail, FIG. 12 A shows an example of the distribution characteristics for a driving condition 
such that a vehicle is entering a freeway, so that the sky is displayed all over the upper section of the 
monitor screen. 

FIG. 12B shows another example of the distribution characteristics for a driving condition such 
5 that the vehicle is travelling on a road uncovered with snow, which is displayed in the mid section of the 
monitoring screen, while the ground besides the road has been covered with snow. 

On the other hand, FIGS. lOA and lOB show the distribution characteristics of the addition of 
luminance Ai while the cameras 1 and 2 is facing a weak sunlight during the sunlight-facing driving. 

The weak-sunlight-facing driving results in a tendency in that high luminance portions being 
1 0 converged on one section corresponding to the sunlight-diSiision disc (FIG. 2) of a monitored image in the 
horizontal direction. This is shown in FIG. 13A in which the high luminance portions are converged on 
the mid section of the monitor screen. FIG. 13B shows high luminance portions converged on the right 
section of the monitor screen. 

Step S6 determines whether the present driving condition is a regular one such as shown in FIG. 
15 1 2 A or 1 2B or the weak-sunlight-facing driving such as shown in FIG. 1 3 A or 1 3B. 

For this determination, a luminance center LC and a luminance moment LM are introduced as 
defined in the following equations: 

LC= Z (ixAi)/Z Ai 

IM= 1 (I i-LC| xAi) ... (2) 

20 where "i" indicates a monitoring area Ni- number (0 to 31) and Ai indicates an addition of the 

luminance Ai in the i-th monitoring area Ni, like in step S5. 
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The luminance center LG is defined as a horizontal position at which the addition of luminance 
Ai in the horizontal direction are converged. The LC is obtained by dividing the total of the product of Ai 
X "i"(=horizontal position) by the total of Ai. 

For example, when all of the addition of luminance Ai are the same value as shown in FIG. 14A, 
5 the luminance center LG is 15. 5 that lies between 0 and 31. Moreover, when the addition of luminance 

AO to A15 are all zero and A16 to A3 1 are the same value 0), the luminance center LG is 23. 5. 

The luminance moment LM is defined as the total of the product of a distance fi*om the 
luminance center LC to a horizontal position "i" times the addition of luminance Ai. The moment LM 
indicates the distribution of the addition of luminance Ai in the horizontal direction on the basis of the 
1 0 luminance center LG. The luminance moment LM becomes small when the addition of luminance Ai is 
small in overall or the luminance is converged in the vicinity of the luminance cento* LG. 

Calculated next is the value obtained by dividing the luminance moment LG by the total of 
addition of luminance 

Z Ai, or luminance moment LM' that is normalized by Z Ai as indicated by the following 

1 5 equation: 

LM' = LM/Z Ai 

- Z (I i-LCl xAi) Z Ai...(3) 

In normal driving at the distribution characteristics such as shown in FIG. 14A, the luminance 
moment LM tends to become large because luminance are distributed all over in the horizontal direction. 
20 On the other hand, in the sunlight-facing driving at the distribution characteristics such as shown 

in FIG. 14G, the luminance moment LM tends to become small because the luminances are converged in 
the vicinity of the luminance center LG. 
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However, it is sometimes difficult to discriminate the sunlight-facing driving condition fiom a 
normal driving condition because the luminance moment LM becomes large for a bright image even in 
sunlight-facing driving. 

In view of this, the normalized luminance moment LM' diat is obtained by dividing the 
5 luminance moment LM by the total of addition of luminance Ai and that depends only on luminance 
distribution is used for the detemiination in stq) S6. 

The normalized luminance moment LM' has a tendency of becoming large for the distribution 
characteristics shown in FIGS. 12A and 12B but small in FIGS. 13A and 13B. 

In view of this, step S6 compares the normalized luminance moment the LM' with an ^propriate 
1 0 reference value for detecting the weak-sunlight-facing driving condition that is shown in FIGS. 13A and 
13B. 

The equation (3) can be modified for a large weight of the addition of luminance Ai for 
evaluation of Ai distribution as follows: 

LM'= I (I i-LC| xAi^ I Ai 

15 LC=I(ix-AiV I Ai...(4) 

In FIG. 5, when all the requirements in steps SI to S6 are met, or all the fail-safe 
measure-intermption requirements are met, it is determined that the vehicle is facing a weak sunlight. A 
count flag Font is then set at "1" (step S12). The count flag Fcnt is set at "0" when a counter K reaches 

the lower limit value 0 whereas " 1 " while the counter K is counting up or down (K ^ 0). 

20 The counter K is set at a count value a (> 0) in the following step S13. 

The counter K is used for a failure determining routine such as shown in FIG. 15 for determining 
whether to take fail-safe measures. 



17 

When the counter K reaches the upper limit value Klim (step S30) due to increment in the 
weak-sunlight-facing driving condition determining routine, a failure flag NG is set at (step S31). On 
the other hand, when the counter K reaches the lower limit value 0 (step S32), the count flag Fcnt and the 
failure flag NG are both reset at "0" (steps S33 and S34). 
5 Moreover, when the counter K is in the range from the lower limit value 0 to the upper limit 

value Klim , the rautine ends via steps S30 and S32 with no change in the count flag Fcnt and the failure 
flagNG. 

While a front-view monitoring is intemipted by the fail-safe measures (NG = 1), it will not be 
resumed (fail-safe measures will not be released) unless the counter K reaches the lower limit value 0. 
10 On the other hand, while the front-view monitoring is performed (NG = 0), it will not be 

intermpted by the fail-safe measures unless the counter K reaches the upper lower hmit value Klim . 

The counter K starts the count-up when a fail-safe measure-intermption requirement is met. 
The failure flag NG is set at "1" when such a driving condition continues for a predetennined period 
(decided based on the upper limit value Klim and the count-up amount D a ) for taking the fail-safe 
1 5 measures. Monitoring is thus intermpted for avoiding a decrease in monitoring control accuracy in the 
weak-sunlight-facing driving. 

The count-up amount a is decided according to a count-up amount setting routine shown in 

FIG. 6. The amount a is decided in accordance with a distance variable and a difference in luminance 

in lefl: and right cameras (step S20). 
20 Usually, cameras are facing the sunlight in the sunlight-facing driving condition and hence the 

variation in calculation of the distance tends to be larger compared to a normal driving condition. 
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In view of such a tendency, the present invention obtains the distance variable in a predetennined 
area in a monitored image, for example, an area extending in the vertical direction in the image. The 

count-up amount a is set at a large one when the variable is large. It is also set at a large amount in the 

sunlight-facing driving condition due to a larger luminance difference between the left and right cameras 
5 compared to the normal driving condition. 

On the other hand, it is determined that the vehicle (the cameras 1 and 2) is not facing the 
sunlight if any one (or more) of the fail-safe measure-intenuption requirements is (or are) not met through 
steps SI to S6 in FIG. 5. 

The routine then proceeds to step S3 in FIG. 6 to determine whether the count flag Fcnt has been 
1 0 set at "1", or the counter K has been counting up or down in step S7. If it is detennined as K = 0, the 
routines will not proceed anymore and ends. 

As disclosed, monitoring is continued without taking any fail-safe measurements during the 
normal driving condition without facing sunlight and the count K having reached the lower limit value 0, 
or the failure flag NG having been set at "0". 
15 On the contrary, when any one (or more) of the fail-safe measure-interruption requirements is (or 

are) not met thrcugh steps SI to S6, but it is ''YES" (Fcnt=l, K 0) in step S7, the routine proceeds to 

step S8. 

Steps S8 to SIO define fail-safe measure-release requirements for the counter K to count up or 
down to resume fiinctions that have been intermpted by the fail-safe measures. The counter K will count 
20 down when a fail-safe measure-release requirement is met but will count up when a requirement is not 
met. 
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The fail-safe measure-release requirements are not so severe than the fail-safe 
measure-intermption requirements for avoiding repetition of the fail-safe measures when it is repeated that 
the vehicle travelling ahead is lost and appeared on the monitoring screen. 

As one of the fail-safe measure-release requirements, it is determined in step S8 whether the 

5 shutter speed "a" is 3000 [J sec. or lower. If not, or this fail-safe measure-release requirement is met, the 

routine proceeds to step S 1 1 in which the counter K counts down by a count-down amount p . 

On the other hand, if "YES" in step S8, the routine proceeds to step S9 to detennine whether the 
luminance-addition variance VAR is equal to or larger than the reference value. 

The VAR tends to be large in the sunlight-facing driving condition because, particularly, when 
1 0 facing a strong sunlight, luminance-saturated sections appear on the monitor screen due to smears or 
sunlight-diSlision discs (FIG. 2) and also blackout sections. 

However, VAR tends to be large not only in the sunlight-facing driving, but also, for instance, in 
a sunny day, due to a strong contrast between sunny and shade places. It tends to be large, particularly, 
there is the shade of guard rails or roadside trees along the road. 
1 5 In view of such a tendency, the present invention deploys the luminance-addition variance VAR 

that will not vary so much in the weak-sunlight-facing driving compared to the normalized luminance- 
addition variance VAR', for the fail-safe measure-release requirements less severer than the fail-safe 
measure-intermption requirements, to avoid the repetition of fail-safe measures discussed above. 

If *^S" in step S9 (FIG. 6), the routine proceeds to step S13 (FIG. 5) in which the counter K 
20 starts count-up. On the other hand, if "NO*' in step S9, the routine proceeds to step SIO to detennine 
whether lane markings have been accurately recognized. Accuracy on a lane marking recognition can be 
used as a fail-safe measure-release requirement for avoiding the repetition of the fail-safe measures 
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discussed above. This is because the road surface will be mostly correctly displayed on the monitor 
screen and hence the lane marking recognition accuracy will not be lowered so much even in the 
weak-sunlight-facing driving condition. 

If the lane marking recognition accuracy has been lowered in step S 10, the routine proceeds to 
5 step S 1 3 (FIG. 5) in which the counter K starts count-up. 

On the other hand, if the lane marking recognition accuracy is not lowered in step S 10, the 
routine proceeds to step S 1 1 in which the counter K starts count-down. 

The lane marking recognition accuracy can be determined as disclosed in Japanese Patent 
Application No. 1 1-269578. The accuracy of lane markings detected on a monitored image is determined 
1 0 based on the number of edges of luminance on the lane markings, the positional relationship (continuity) to 
the lane markings detected previously and parallelism between left and right lane markings. High lane 
marking recognition accuracy is achieved as the number of edges of luminance is larger, the lane markings 
detected in the present image more continues to the previously detected maridngs and the detected left and 
right markings are more parallel to each other. 
1 5 As clearly understood fiom the foregoing disclosure, the fail-safe measure-release requirements 

(count-down requirements for the counter K) are as follows: 

(1) the shutter speed "a" is higher than 3000 |J se.,or 

(2) if not, the luminance-addition variance VAR is smaller than the reference value and lane 
marking recognition is accurate. 

20 As disclosed, the present invention monitors parameters such as the luminance-distribution 

characteristics parameter, the number of edges of luminance, the upper and lower luminance saturation 
factors and the luminance moment for detection of a weak-sunlight-facing driving condition condition. 
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The fail-safe measures can therefore be taken exactly during the weak-sunlight-facing driving 
condition, thus oflFering a highly reliable vehicle fiont-view monitoring system. 

Particularly, in the present invention, several monitoring areas are set on the monitor screen, 
which are slit-like areas extending vertically on a monitored image for obtaining the luminance-distribution 
5 characteristic values (the luminance-addition variance VAR and the maximum addition of luminance 
Amax) in the horizontal direction of the image. 

The luminance-distribution characteristic values are normalized using a shutter speed "a" for 
obtaining parameters (the luminance-addition variance VAR' and the normalized maximum addition of 
luminance Amax*) that are highly related to weak-sunlight-facing driving condition. 
1 0 The normalized parameters VAR' and Amax' can be used for detection of a weak-sunlight-facing 

condition, which is more accurate than using the un-normalized parameters VAR and Amax as the 
fail-safe measure-intenuption requirements. 

The fail-safe measure-release requirements are set as including several conditions in addition to a 
weak-sunlight-facing driving condition so that it is not as severe as those of the fail-safe 
1 5 measure-intermption requirements. 

This requirement setting avoids the repetition of unnecessary fail-safe measures such as the 
warning which could initate the driver while it is repeated that the vehicle traveUing ahead is not visible 
and appeared on the monitoring screen. 

The fail-safe measure-intermption requirements include the normalized Luminance-addition 
20 characteristic values VAR' and Amax ' whereas the fail-safe measure-release requirements include VAR 
and Amax, the parameters different &om VAR' and Amax because those are not normalized. 

Moreover, the fail-safe measure-release requirements include the accuracy on the lane marking 
recognition that is not deployed for the fail-safe measure-intenuption requirements. 
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Deploying different parameters for the fail-safe measure- interruption/release requirements offers 
flexible fail-safe requirement settings that match an actual driving condition compared to setting in which 
reference values are changed between the intermption/release requirements for the same parameters. 

The count-up amount a is adjusted based on the luminance difference between the left and 

5 right cameras and distance data variable. 

This adjustment offers variable settings for a fail-safe determination time fix)m a moment at 
which the fail-safe measure-intermption requirements are met to a moment of taking fail-safe measures in 
accordance with actual driving conditions. 

Disclosed so far is the vehicle front-view monitoring system that takes fail-safe measures against 
1 0 a weak-sunlight-driving condition in which the vehicle is travelling facing the weak sunlight so that 
sunlight-diflRision discs ^pear on the monitor screen. 

However, the present invention is ^plicable to not only such a weak-sunlight-driving condition 
but also another optical phenomenon such as smear occurring on a monitoring screen. 

As disclosed, according to the present invention, optical irregularities occurring on the monitored 
1 5 images due to optical interfering can be accurately detected, thus, the fail-safe measures can be taken 
swiftly in response to such an optical phenomenon for insuring safe driving. 

Moreover, different parameters for the fail-safe measure-intermption/release requirements offer 
flexible fail-safe requirement settings that match actual driving conditions. 

It is fiirther understood by those skilled in the art that the foregoing description is a prefen-ed 
20 embodiment of the disclosed device and that various change and modification may be made in the 
invention without departing from the spirit and scope thereof 



